Skip to content

Use managed NTLM on RHEL 8 to fix NegotiateAuthentication test failures#129468

Open
medhatiwari wants to merge 1 commit into
dotnet:mainfrom
medhatiwari:fix-ntlm-rhel8
Open

Use managed NTLM on RHEL 8 to fix NegotiateAuthentication test failures#129468
medhatiwari wants to merge 1 commit into
dotnet:mainfrom
medhatiwari:fix-ntlm-rhel8

Conversation

@medhatiwari

Copy link
Copy Markdown
Contributor

Summary

RHEL 8 ships gss-ntlmssp 1.2.0 built against OpenSSL 1.1.1. Unlike RHEL 9/10 (which use OpenSSL 3.x), this combination causes gssntlmssp to fail when producing NTLM Type 3 (Authenticate) messages, GetOutgoingBlob returns GenericFailure when processing a server challenge.

This is the same class of issue already handled for Ubuntu 24/26 and openSUSE 16 (which ship a broken gss-ntlmssp 1.2). The fix adds RHEL 8 to the UseManagedNtlm condition using PlatformDetection.IsRedHatFamily && !PlatformDetection.IsOpenSsl3, which enables the managed NTLM implementation only on RHEL versions with OpenSSL < 3 (i.e., RHEL 8). RHEL 9 and 10 (OpenSSL 3.x) are unaffected.

Verification

Tested on a RHEL 8.10 s390x machine (gssntlmssp-1.2.0-1.el8_8, OpenSSL 1.1.1k):

  • Without managed NTLM: Client Negotiate succeeds, but processing the server challenge returns GenericFailure — gssntlmssp cannot produce an Authenticate message.
  • With managed NTLM (DOTNET_System_Net_Security_UseManagedNtlm=1): Full NTLM handshake completes successfully — valid Type 3 message (356 bytes, status=Completed).

This resolves 11 NegotiateAuthentication test failures seen in CI on RHEL 8.

@dotnet-policy-service dotnet-policy-service Bot added the community-contribution Indicates that the PR has been added by a community member label Jun 16, 2026
@dotnet-policy-service

Copy link
Copy Markdown
Contributor

Tagging subscribers to this area: @dotnet/ncl, @bartonjs, @vcsjones
See info in area-owners.md if you want to be subscribed.

@rzikm rzikm left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area-System.Net.Security community-contribution Indicates that the PR has been added by a community member

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants